How Transparency Policies Shape the Global Influence of Authoritarian Regimes on Social Media

Allison Koh

Centre for Artificial Intelligence in Government at

About Me

Research Fellow, Centre for Artificial Intelligence in Government @ University of Birmingham
PhD from the Hertie School’s Centre for International Security, Berlin
Research Foci
- Geopolitics of emerging technologies
  - How social media serves the foreign policy interests of authoritarian regimes
- Emerging technologies as a research tool for studying conflict
  - Event extraction with LLMs: identifying allegations of state repression/human rights violations from multiple sources of text data
  - LLM agents to identify relationships between actors in conflict settings; foreign interference in elections

Today

Did the reversal of Twitter’s transparency policies increase the influence of authoritarian regimes?

Comparative case study: Official Government Accounts on Post-Musk Twitter

#TBT

“We believe that people have the right to know when a media account is affiliated directly or indirectly with a state actor.”

– Twitter 2020

#TBT

Twitter, ca. August 2020 - March 2023

#TBT

The “After Times” of Twitter

March 29, 2023

The “After Times” of Twitter

April 21, 2023

Who was affected by these policy shifts?

Official government and diplomatic accounts

Voices of the nation-state abroad
Highly visible regardless of labels

State-affiliated media

Not editorially independent from the state
Less attributable to the state without labels

So who was affected by these policy shifts?

Two types of state actors were affected by these policy shifts: First, on the left hand side, // official government and diplomatic accounts represent voices of the nation-state abroad, // and are highly visible as individuals and entities connected to their respective governments regardless of whether the platform they’re on chooses to add transparency markers.

On the other hand, state-affiliated media which are outlets // that are not editorially independent from the state do not represent such a visible role. They’re meant to promote narratives that positively contribute to their government’s international reputation, // but without platform-assigned labels, they’re not as attributable to the state and can pass as independent outlets.

Theoretical Expectation

Erosion of transparency policies

⬇️

State actors took advantage of new opportunities for authoritarian image management¹.

⬇️

They gained influence on the platform.

Expectations

💡 General Hypotheses

After each reversal of Twitter’s transprency policies…

Feature	Hypotheses
Engagement	Reposts ⬆️; Likes ⬆️
User Behavior	Post frequency ⬆️; Topic diversity ⬇️

🔎 Expectations: State Actor Types

State actors with high visibility will be less responsive to platform changes.

🏦 Official government accounts: null/less pronounced effects
📺 State-affiliated media: more pronounced effects

Exploratory Analysis

🌏 Country

Do we observe differences in online behavior/engagement across countries?

Data

🏦📺 State Actors: Hamilton 2.0 Dashboard¹

1,177 accounts linked to Chinese, Iranian, and Russian state actors

Proxy for accounts that had government and state-affiliated media labels

Data

Twitter Academic API via `twarc` CLI

Tweet timeline endpoint; up to ~3,200 tweets per account

Collected ~2.5 million tweets from 1,038 timelines

Measurement

📈 Variables

DV: Online activity of/with state actors
- Engagement: Retweets, likes
- Behavior: Tweets, topic entropy

IV: Policy-affected status
- Algorithm change: Media accounts
- Label removal: All state-affiliated media

📏 Measuring Topic Entropy

Shannon Entropy of topic distributions per user-day (Shannon 1948)
- Calculated using a multilingual BERTopic model
- Lower entropy ~ more focused discussion

💻 Method

Interrupted Time Series Analysis
Robustness Checks: Generalized Synthetic Control (Xu 2017), RDD

Results: Actor Type

⬆️ engagement with media; 🕸️ consolidation of messaging after label removal

Average per-user metrics

Results: Actor Type + Country

⬆️ engagement with Russian accounts; \(\Delta\) user behavior of Chinese + Russian accounts

Average per-user metrics

Takeaways

Findings

After Twitter’s erosion of transparency policies for state actors…

Notable increases in engagement with media after 2 major policy changes
Messaging was (slightly) consolidated after label removal
Of the three countries, Russia benefited the most from these policy changes¹.

Next Steps

Finalizing model specs
What to make of the findings for content analyses?

Thank you!

a.w.koh@bham.ac.uk

https://allisonkoh.github.io/

🦋 @allisonwkoh

Appendix

Twitter: A safe space for state influence?

Speaking of Elon Musk, after his acquisition of Twitter, the company became more vulnerable to interference from state actors. And when governments are able to have this much influence over content removal, it affects the free flow of information across borders.

To the left: We see that Twitter has been getting more court orders and government requests to remove content and hand over private data on individual users.

As shown on the right Between October 2022 and April 2023, Twitter fully complied with around 80% of legal demands from governments and courts. This was a definitive increase compared to how legal demands were handled in the past; in 2021, the compliance rate for such requests was approximately 50% (Twitter Transparency Center, 2022).

In addition, Twitter has also limited access to information on legal demands and other platform activity from researchers and third-party organizations. (and many of us in this room have very likely been affected by this)

Twitter’s Policy Changes in April 2023

**March 29:** ⬆️ engagement with prominent state media from China, Russia, Iran (Kann 2023)

**April 6:** ~~“In the case of state-affiliated media entities, Twitter will not recommend or amplify accounts or their Tweets with these labels to people.”~~

**April 12:** NPR quits Twitter after being labeled as “state-affiliated media” (Folkenflik 2023)

**April 21:** Twitter removes all labels for “government” and “state-affiliated media”

Many of Twitter’s policy changes concerning state actors, which ultimately led to the removal of relevant profile labels, rapidly escalated earlier this year.

On March 29th, The Atlantic Council’s Digital Forensics Lab identified increased engagement in terms of follows and likes, which signified a change in Twitter’s policy that stated “in the case of state-affiliated media entities, Twitter will not recommend or amplify accounts or their tweets with these labels to people”. Shown here: Average Twitter followers for Russia Today. On March 28th, when X-axes Jan 1 2023 -> sometime mid-April 2023

On April 6th, this clause was officially removed from Twitter’s Platform Use Guidelines.

Throughout April, Twitter started flagging outlets from the US, UK, and Canada as “state-affiliated” or “government-funded” media, going against their policy to not label public broadcasters that are editorially independent. This move essentially obscured the original intent for the labels to serve as a flag for politically-motivated content, and led NPR to quit the platform on April 12th.

After all this controversy over profile labels and whether public broadcasters from democracies with media freedom should be tagged with similar labels as state-affiliated media that follow the directive of their governments, Twitter removed all labels for “government” and “state-affiliated media” altogether.

Research Questions and Definitions

❓ Research Questions

After Twitter’s removal of profile labels on April 21st, 2023…

…did engagement with state actors increase?
…did state actors change how they use the platform?

📖 Definitions¹

State actor: Any individual or entity connected to government or state-affiliated media
Government official: Key individuals/entities representing “voices of the nation state abroad”
State-affiliated media: Outlets where the state exercises control over editorial content

Data on State Actors

Hamilton 2.0 Dashboard¹

1,177 accounts linked to Chinese, Iranian, and Russian state actors

Proxy for accounts that had government and state-affiliated media labels

Measurement

Dependent variables

Per-day metrics 📈

Retweets and likes of tweets authored by state actors (H1)
Tweet volumes and “focus”¹ of posts produced by state actors (H2)

Independent variables

State actor type (official government or state-affiliated media)
Time-related variables for parametric tests

Results: Actor Type

⬆️ engagement with media; 🕸️ consolidation of messaging after label removal

Daily-aggregated totals/averages

Results: Actor Type + Country

⬆️ engagement with Russian accounts; \(\Delta\) user behavior of Chinese + Russian accounts

Daily-aggregated totals/averages

Results - Placebo Tests

Null results at the aggregate level

Placebo Tests—Engagement and Tweet Volume

Results - Synthetic Control

All State Actors

Results - Synthetic Control

State Actor Type

Results - Synthetic Control

State Actor Type + Country

References

Dukalskis, Alexander. 2021. Making the World Safe for Dictatorship. Oxford University Press.

Folkenflik, David. 2023. “NPR Quits Twitter After Being Falsely Labeled as ’State-Affiliated Media’.” NPR, April.

Kann, Alyssa. 2023. “State-Controlled Media Experience Sudden Twitter Gains After Unannounced Platform Policy Change.” DFRLab.

Munger, Kevin, Richard Bonneau, Jonathan Nagler, and Joshua A. Tucker. 2019. “Elites Tweet to Get Feet Off the Streets: Measuring Regime Social Media Strategies During Protest.” Political Science Research and Methods 7 (04): 815–34. https://doi.org/10.1017/psrm.2018.3.

Schafer, Bret. 2019. “Hamilton 2.0 Methodology & FAQs.” Alliance For Securing Democracy.

Shannon, Claude Elwood. 1948. “A Mathematical Theory of Communication.” The Bell System Technical Journal 27 (3): 379–423.

Twitter. 2020. “New Labels for Government and State-Affiliated Media Accounts.” https://blog.twitter.com/en_us/topics/product/2020/new-labels-for-government-and-state-affiliated-media-accounts.

Xu, Yiqing. 2017. “Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models.” Political Analysis 25 (1): 57–76. https://doi.org/10.1017/pan.2016.2.

How Transparency Policies Shape the Global Influence of Authoritarian Regimes on Social Media

About Me

Today

#TBT

#TBT

#TBT

The “After Times” of Twitter

March 29, 2023

The “After Times” of Twitter

April 21, 2023

Who was affected by these policy shifts?

Theoretical Expectation

Erosion of transparency policies ⬇️ State actors took advantage of new opportunities for authoritarian image management1. ⬇️ They gained influence on the platform.

Expectations

💡 General Hypotheses

🔎 Expectations: State Actor Types

Exploratory Analysis

🌏 Country

Data

🏦📺 State Actors: Hamilton 2.0 Dashboard1

Data

Twitter Academic API via twarc CLI

Measurement

📈 Variables

📏 Measuring Topic Entropy

💻 Method

Results: Actor Type

Results: Actor Type + Country

Takeaways

Findings

Next Steps

Thank you!

Appendix

Twitter: A safe space for state influence?

Twitter’s Policy Changes in April 2023

Research Questions and Definitions

❓ Research Questions

📖 Definitions1

Data on State Actors

Hamilton 2.0 Dashboard1

Measurement

Dependent variables

Independent variables

Results: Actor Type

Results: Actor Type + Country

Results - Placebo Tests

Null results at the aggregate level

Results - Synthetic Control

All State Actors

Results - Synthetic Control

State Actor Type

Results - Synthetic Control

State Actor Type + Country

References

Erosion of transparency policies

⬇️

State actors took advantage of new opportunities for authoritarian image management¹.

⬇️

They gained influence on the platform.

🏦📺 State Actors: Hamilton 2.0 Dashboard¹

Twitter Academic API via `twarc` CLI

📖 Definitions¹

Hamilton 2.0 Dashboard¹